|
|
|
The Hierarchical Data Format (HDF) has been in existence since the 1980's. Over the years, HDF has been chosen as the format of choice for a number of large programs, including NASA's Earth Observing System (EOS) and now NASA/NOAA/DOD's National Polar-orbiting Operational Environmental Satellite System (NPOESS). As a result, data centers scattered around the world currently archive several petabytes of data stored in one or another flavor of the HDF format - more data accumulates each and every day. Yet HDF was never meant to be a format for long-term archival. Moreover, HDF was never meant to be used without the accompanying HDF libraries that allow users to read or write data in HDF format without a detailed understanding of the physical data layout.
As time progresses so does technology. The HDF4 of yesteryear becomes the HDF5 of today and the HDF6 or HDF7 of tomorrow. Yet it is highly unlikely that the contents of existing archives will be migrated from format to format as technology progresses. It is equally hard to envision that maintenance of each and every version of the needed HDF libraries will continue into the indefinite future. This presents a dilemma for centers with the mandate to preserve and make accessible these data for the long term. This presentation describes the issue in depth and solicits community input on proposed solutions.
Back to Agenda